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Abstract 

Predicting  a  leader’s  actions  must  take  the  Subject’s  personality  into  consideration  in  addi¬ 
tion  to  relevant  situational  variables.  This  paper  presents  a  methodology  for  prediction  that 
enables  the  analyst  to  reason  through  a  prediction  of  a  Subject’s  decision  making,  to  identify 
assumptions  and  determinant  variables,  and  to  quantify  each  variable’s  relative  contribution 
to  the  prediction,  producing  a  graphical  representation  of  the  analysis  with  explicit  levels  of 
uncertainty.  The  analyst  builds  Bayesian  networks  that  integrate  situational  information  with 
the  Subject’s  personality  and  culture  to  provide  a  probabilistic  prediction  of  the  hypothesized 
actions  a  Subject  might  choose.  The  model  development  process  allows  the  analyst  to  sys¬ 
tematically  develop  hypotheses  regarding  potential  actions,  determine  the  Subject’s  most 
likely  strategic  objectives,  identify  relevant  situational  variables,  estimate  probabilistic  rela¬ 
tionships  between  variables,  and  assess  the  Subject’s  standing  on  several  personality  vari¬ 
ables.  The  analysis  forecasts  probabilities  of  hypothesized  actions,  which  may  then  be  sub¬ 
jected  to  several  “what-if”  and  sensitivity  analyses.  The  methodology  has  been  applied  to 
over  a  dozen  historical  and  prospective  situations. 

Introduction 

One  generalization  that  can  be  made  about  intelligence  analysis  is  that  it  involves  ac¬ 
tivities  in  which  members  of  one  group  of  people  -  analysts  -  make  predictions  about  the  ac¬ 
tivities  of  members  of  another  group  of  people  -  foreign  leaders  or  other  decisionmakers. 
The  decisionmaker  determines  the  course  of  action  to  take,  considering  his  or  her  objectives 
as  well  as  a  variety  of  situational  constraints  in  a  way  that  reflects  his  or  her  personality  as 
well  as  cultural  norms.  The  analyst  must  identify  the  relevant  situational  variables,  specify 
the  personal  characteristics  of  the  decisionmaker,  and  forecast  how  these  factors  interact  to 
determine  the  chosen  action. 

Personal  characteristics  of  the  analyst  may  make  it  more  or  less  difficult  to  produce 
an  accurate  prediction.  Many  cognitive  limits  and  biases  in  prediction  are  both  well-known 
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and  widely  shared  among  both  trained  and  naive  analysts.  These  biases  -  such  as  recency, 
halo,  proximity,  hindsight,  and  personalization  -  can  produce  systematic  errors  in  predic¬ 
tions.  In  addition,  prediction  is  often  undertaken  by  a  team  of  analysts  with  diverse  back¬ 
grounds,  working  on  a  problem  that  will  evolve  over  weeks  or  months  and  need  periodic  re¬ 
porting.  Consequently,  the  analysis  process  is  subject  to  less  widely  discussed  but  influential 
social  biases,  such  as  those  reflected  in  giving  undue  weight  to  the  senior  expert,  the  “party 
line”  or  published  record,  the  analyst  with  the  biggest  fistful  of  cables,  or  with  the  most  daz¬ 
zling  personality. 

This  paper  describes  an  effort  to  develop  tools  that  can  assist  analysts  in  making  rea¬ 
soned  predictions  about  what  key  figures  might  do  in  specific  situations.  This  effort,  under¬ 
way  since  2001,  addresses  many  of  the  concerns  in  recent  critiques  of  the  national  intelli¬ 
gence  process.  The  key  purpose  of  our  work  is  to  create  a  method  for  modeling/predicting 
leader  actions. 

The  following  principles  were  the  basis  of  this  development  effort: 

•  Viewpoints  -  the  more  the  merrier,  but  make  it  systematic  (combine  many  analysts  with 
varying  expertise,  addressing  both  the  situation  and  the  subject’s  personality  and  cul¬ 
ture,  and  use  a  rigorous  analytical  method  for  integration); 

•  Intelligence  as  a  process,  not  just  as  product  -  recognizing  that  intelligence  consumers 
are  their  own  analysts,  engage  the  customer  by  using  the  model  to  manage  the  debate 
and  questioning  that  often  ensues  when  an  analyst  briefs  a  policy-maker  -  the  modeling 
process  enables  different  assumptions  or  alternative  hypotheses  to  be  tested  on  the  spot; 

•  Continuous  and  real-time  updating  of  the  model  -  review  and  quantify  relevant  evi¬ 
dence  and  the  associated  probabilities  for  specific  model  variables,  and  explicitly  in¬ 
form  the  user  when  data  may  warrant  changing  judgments. 

The  paper  describes  the  methods  we  have  used  to  predict  leader  actions,  based  on  a 
combination  of  situational  variables  and  an  assessment  of  the  leader’s  personality.  First,  it 
describes  the  Bayesian  network  methodology  that  forms  the  basis  for  forecasting  leader  ac¬ 
tions.  This  methodology  specifically  addresses  some  of  the  individual  cognitive  biases  that 
can  degrade  the  quality  of  predictions.  Second,  the  paper  describes  some  of  the  most  relevant 
research  results  that  link  a  leader’s  personality  to  his  or  her  actions  and  shows  how  these  can 
be  represented  in  the  Bayesian  framework.  Third,  it  describes  the  process  that  we  have  used 
with  groups  of  analysts  to  develop  Bayesian  models  to  understand  historical  events  and  to 
predict  future  actions.  We  believe  that  this  development  process,  along  with  the  underlying 
methodology,  can  reduce  the  impact  of  social  biases  on  the  accuracy  of  predictions.  Finally, 
the  paper  summarizes  the  status  of  our  work  and  describes  some  future  directions  that  it  may 
take. 
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Modeling  Method 


Forecasting  is  mankind’s  attempt  to  guess  at  what  the  future  will  be  like.  Domains  in 
which  forecasting  is  important  involve  the  weather  for  farming  and  human  activities;  business 
ventures  such  as  how  well  will  our  new  product  do  and  when  will  competition  launch  a  new 
product,  how  much  will  it  cost,  and  how  will  it  compare  to  existing  products;  and  governmental 
activities  such  as  what  is  an  enemy  doing  and  how  much  will  a  new  social  program  cost  and  how 
will  it  perform.  Mankind  has  used  the  stars  (astrology),  experts  (subject  matter  experts  and 
soothsayers),  and  mathematical  statistics  to  perform  forecasting.  There  is  great  room  for  im¬ 
provement  in  both  the  art  and  science  of  forecasting. 

Common  scientific  methods  for  forecasting  include  role  playing,  obtaining  and  combin¬ 
ing  expert  opinions,  using  analogies,  extrapolating  from  historical  data,  building  rule-based  mod¬ 
els  that  operate  on  recent  and  current  data,  using  statistical  models  that  involve  time  series  or 
cross-sectional  data,  and  various  combinations  of  these  methods.  Armstrong  (2001)  provides  the 
methodology  tree  of  possible  forecasting  methods  shown  in  Figure  1.  Our  use  of  Bayesian  net¬ 
works  described  in  this  paper  fall  under  judgmental  -  using  others  -  structured  -  limited  feed¬ 
back  (Delphi,  Decomposition,  Judgmental  bootstrapping). 


Figure  1.  Tree  of  Forecasting  Methods  (from  Armstrong  [2001]) 


The  general  problem  of  predicting  someone’s  future  action  is  exceedingly  complex. 
Without  even  considering  the  task  of  identifying  the  determinative  variables  correctly,  one 
must  deal  with  uncertainty,  human  judgment  about  the  problem  logic,  relative  strength  of 
specific  variables  and  evidence,  and  the  dependencies  of  some  variables  on  others.  When  we 
add  the  requirement  to  enable  updates  to  the  prediction  as  new  information  becomes  avail- 
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able,  we  realize  there  is  only  one  method  that  matches  the  problem  statement  -  Bayesian 
probability  (Schum  1994). 

Recent  advances  in  computer  science  and  operations  research  have  created  Bayesian 
networks  (Pearl  1988;  Shachter  1988).  Bayesian  networks  provide  a  graphical  representation 
of  the  problem,  using  an  acyclic  directed  graph  to  show  the  variables  as  nodes,  the  probabil¬ 
istic  dependence  as  arcs,  and  probabilistic  independence  as  the  lack  of  arcs.  Conditional 
probabilities  are  captured  inside  the  nodes.  Sophisticated  message  passing  algorithms  are 
used  to  update  the  probabilities  at  all  nodes  based  on  evidence  at  several  nodes  (Buede 
2001). 

Application  of  this  modeling  method  provides  a  natural  mechanism  for  surfacing  as¬ 
sumptions,  logic,  and  new  evidence  for  the  team  working  the  problem.  In  addition,  modeling 
software  can  capture  an  auditable  history  of  the  team’s  thought  process  and  supporting  evi¬ 
dence.  This  software  solution  alerts  the  analyst  when  certain  thresholds  are  met  within  the 
model,  indicating  that  the  evidence  may  warrant  changing  one’s  beliefs. 

There  is  some  similarity  between  this  approach  and  Alternative  Competing  Hypothe¬ 
ses  (ACH;  Heuer  1999).  In  both  our  approach  and  ACH,  a  list  of  possible  hypotheses  is  de¬ 
veloped.  Next,  a  set  of  possible  indicators  is  brainstormed;  these  indicators,  if  true,  would 
favor  one  hypothesis  over  the  other.  At  this  point  the  methods  diverge.  The  ACH  elicits 
qualitative  statements  (e.g.,  1  to  3  pluses  or  1  to  3  minuses)  to  capture  the  strength  of  the  re¬ 
lationship  between  the  hypothesis  and  each  indicator.  The  Bayesian  approach  quantifies  the 
conditional  probability  of  the  indicator  given  each  of  the  hypotheses.  With  ACH,  the  result  is 
a  summary  of  the  pluses  and  minuses  associated  with  each  hypothesis  for  identified  indica¬ 
tors.  The  result  of  a  Bayesian  network  is  the.  posterior  probability  of  the  hypotheses  given  the 
identified  indicators.  The  Bayesian  approach  can  also  incorporate  causal  factors  that  condi¬ 
tion  the  probability  of  the  hypotheses,  address  interactions  between  the  indicators,  and  report 
uncertainty  associated  with  the  indicators  and  causal  factors. . 

Another  approach  that  has  seen  lots  of  applications  in  the  last  decade  is  the  Situ¬ 
ational  Influence  Assessment  Module  (SIAM;  Rosen  &  Smith,  1996).  SIAM  is  used  to  com¬ 
pare  alternate  scenarios  of  causal  factors  (rather  than  indicators)  for  creating  some  desired  or 
undesired  outcome  (hypothesis).  A  SIAM  model  looks  like  a  Bayesian  network  in  which 
there  are  many  arrows  entering  a  few  nodes.  In  true  Bayesian  networks,  this  approach  pro¬ 
duces  the  need  for  an  unmanageable  number  of  conditional  probability  distributions.  Using 
approximations  involving  labeling  causal  variables  as  promoters  and  inhibiters,  SIAM  re¬ 
duces  the  number  of  conditional  probability  distributions  to  a  manageable  level.  These  inde¬ 
pendence  assumptions  must  be  considered  carefully  on  a  problem  basis  when  judging  the  ap¬ 
plicability  of  SIAM. 

Representing  Leader  Personality 

A  substantial  body  of  research  recorded  in  both  the  psychology  and  political  science  lit¬ 
erature  has  focused  on  the  cognitive  processes  leaders  use  to  perceive  and  understand  the  politi¬ 
cal  environment,  predict  the  actions  of  adversaries,  generate  and  evaluate  courses  of  action,  make 


4 


and  implement  decisions,  and  receive  and  process  feedback  on  the  outcome  of  these  decisions. 
This  research  has  placed  bounds  on  the  rationality  of  human  actors  (Simon,  1957;  Rosati,  2001) 
and  has  documented  heuristics  and  their  concomitant  biases  in  perception  (Jervis,  1976),  under¬ 
standing  of  uncertainty  (Kahneman,  Slovic,  &  Tversky,  1982),  and  social  judgment  (Nisbett  & 
Ross,  1980).  This  research  indicates  that  a  leader’s  perceptions  of  the  situation  may  be  a  more 
potent  predictor  of  action  than  objective  reality.  It  also  suggests  that  methods  to  forecast  leader 
policy  decisions  should  consider  such  factors  as  beliefs,  cognitive  capacities,  personality,  and 
response  propensities. 

Hermann  (1999)  has  attempted  to  characterize  the  interactions  among  several  personal 
variables  related  to  the  actions  of  political  leaders:  (a)  Belief  that  one  can  influence  or  control 
events,  (b)  need  for  power,  (c)  conceptual  complexity,  (d)  self-confidence,  (e)  task  focus  and 
problem  solving  vs.  relationship  building,  (f)  general  distrust  and  suspicion,  and  (g)  ingroup  bias. 
These  variables  were  combined  to  define  leadership  styles  organized  around  the  following  three 
questions. 

1.  How  do  leaders  react  to  political  constraints  in  their  environment?  Do  the  respect  or 
challenge  them? 

2.  How  open  are  leaders  to  incoming  information?  Conversely,  how  selective  are  they  in 
their  use  of  information  to  guide  their  actions? 

3.  What  are  the  leaders’  reasons  for  seeking  their  positions?  Are  they  driven  infernally  or 
by  relationships  with  their  constituencies? 

For  example,  according  to  Hermann  (1999)  leaders’  reactions  .to  political  constraints  are  deter¬ 
mined  by  their  belief  that  they  can  control  events  and  their  need  for  power.  Individuals  who  are 
high  in  both  of  these  traits  are  more  likely  to  challenge  constraints  that  the  political  environment 
places  on  them.  Conversely,  individuals  who  are  low  on  these  traits  will  tend  to  respect  con¬ 
straints.  Individuals  who  are  high  in  one  trait,  but  low  in  the  other  will  also  tend  to  challenge 
constraints,  but  may  not  be  as  comfortable  or  successful  in  doing  so. 

Psychological  research  has  demonstrated  that  personality  variables  describe  differences 
between  individuals  on  both  cognitive  and  non-cognitive  dimensions.  Although  there  are  many 
potential  attributes  that  can  characterize  individual  differences,  a  substantial  body  of  research  has 
shown  that  five  general  personality  dimensions  can  summarize  most  of  these  traits.  In  a  review 
of  the  literature  on  personality  structure,  Digman  (1990)  summarized  the  growing  concurrence 
within  the  psychological  community  regarding  opinions  concerning  the  structure  of  the  concepts 
of  personality,  as  well  as  in  the  language  of  personality.  Early  efforts  to  define  a  taxonomy  of 
personality  attributes  have  let  to  the  development  of  a  general  structure  that  has  unified  several 
earlier  theories.  The  result  of  this  research,  termed  the  five-factor  model  of  personality,  provides 
general  personal  variables  with  the  potential  to  predict  behaviors  that  are  of  interest  to  this  pro¬ 
ject. 

The  five-factor  model  reflects  the  relationships  among  the  most  commonly  used  psycho¬ 
logical  dimensions.  Factor  analysis  of  these  measures  resulted  in  the  identification  of  a  common 
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five-factor  solution.  These  are  the  five  factors  which  when  taken  together  they  provide  a  good 
approximation  of  what  personality  structure  represents. 

•  Extraversion.  Extraversion  is  characterized  by  a  social  rather  than  a  misanthropic  person¬ 
ality.  The  extravert  is  outgoing  rather  than  introverted  and  expresses  confidence  rather 
than  timidity. 

•  Agreeableness.  This  factor  is  indicated  by  general  friendliness,  rather  than  indifference  to 
others.  The  agreeable  individual  is  docile  rather  than  hostile  and  is  compliant  to  others’ 
wishes. 

•  Conscientiousness.  Conscientiousness  is  the  most  ambiguous  of  the  five  factors.  It  can  be 
seen  as  educational  achievement  or  as  will  or  volition. 

•  Neuroticism.  This  factor  reflects  anxiety  and  dependence,  rather  than  adjustment  or  inde¬ 
pendence.  The  scale  for  this  factor  is  often  reversed  so  that  the  factor  assesses  emotional 
stability. 

•  Openness.  Openness  is  a  reflection  of  an  inquiring  intellect.  Individuals  who  are  high  on 
this  factor  tend  to  be  flexible  and  rebellious,  rather  than  conforming  and  subdued. 

Our  approach  to  account  for  leader  personality  combined  these  two  approaches  from 
political,  science  and  psychology.  After  a  detailed  review  of  the  personality  literature  and  a 
consensus  session  with  some  of  the  leading  researchers,  we  identified  the  following  variables 
from  the  political  psychology  literature:  positive  image  of  others,  internal  locus  of  control, 
need  for  power,  conceptual  complexity,  general  distrust  and  suspicion,  and  acceptance  of 
risk.  (Sticha  et  al.  2000)  From  the  personality  researchers  within  psychology,  the  emphasis 
was  on  the  five-factor  model  (Costa  and  McCrae,  1985).  Our  early  attempts  at  model  build¬ 
ing  demonstrated  that  the  leadership  analysts  were  more  familiar  and  comfortable  with  the 
concepts  from  political  science  than  the  five-factor  model.  However,  the  five-factor  model 
has  substantial  research  backing  it  up,  as  well  as  validated  assessment  instruments.  In  an  ef¬ 
fort  to  synthesize  user  acceptance  and  empirical  foundations,  we  decided  to  integrate  the  two 
sets  of  personality  factors. 

Psychologists  at  HumRRO  related  the  30  facets  from  the  five-factor  model  of  person¬ 
ality  to  the  six  leadership  variables  from  political  science/political  psychology.  The  facets  are 
the  second  tier  elements  of  the  five-factor  model;  each  of  the  five  factors  has  six  facets.  Ta¬ 
ble  1  shows  the  relationships  established  between  the  two  models.  A  minus  sign  (-)  to  the  left 
of  a  facet  indicates  an  inverse  relationship  between  that  facet  and  the  corresponding  leader¬ 
ship  variable. 

The  second  major  element  of  the  personality  model  is  the  incorporation  of  data  and 
associated  error.  There  are  several  ways  to  report  and  assess  data.  The  NEO  (a  commercially 
available  personality  test  with  a  form  for  knowledgeable  informants)  is  a  well-known,  vali¬ 
dated  measure  of  the  facets.  Profiler+  (Young,  2001)  is  a  content-analysis  approach  that  ana¬ 
lyzes  first-person  verbalizations  according  to  Hermann’s  (1984)  personality  theory  of  leader- 
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ship.  Finally,  HumRRO  psychometricians  developed  a  short,  third-party  evaluation  form 
based  on  our  variables.  The  estimated  error  of  each  kind  of  assessment  is  considered  in  the 
model. 


Table  1.  Lin 

cage  Between  Two  Personality  Models 

Political 

Psychology 

Facets  from  5-Factor  Model 

Positive  Image 
of  Others 

Positive  Emotion  (Extraversion) 

Trust  (Agreeableness) 

Internal  Locus 
of  Control 

(-)  Vulnerability  (Neuroticism) 

(-)  Depression  (Neuroticism) 
Assertiveness  (Extraversion) 
Competence  (Conscientiousness) 

S elf-Discipline  (Conscientiousness) 

Need  for  Power 

(-)  Compliance  (Agreeableness) 
Achievement  Striving  (Conscientiousness) 
Assertiveness  (Extraversion) 

Conceptual 

Complexity 

Openness  to  Ideas  (Openness) 
Openness  to  Values  (Openness) 
Openness  to  Actions  (Openness) 

General  Distrust 
&  Suspicion 

(-)  Trust  (Agreeableness) 

Angry  Hostility  (Neuroticism) 

(-)  Warmth  (Extraversion) 

(-)  Compliance  (Agreeableness) 

Acceptance  of 
Risk 

Openness  to  Actions  (Openness) 

(-)  Anxiety  (Neuroticism) 

(-)  Deliberation  (Conscientiousness) 
Excitement  Seeking  (Extraversion) 

(-)  Vulnerability  (Neuroticism) 

The  third  major  element  of  the  personality  model  specifies  how  it  should  be  con¬ 
nected  to  specific  hypotheses  about  leader  actions.  To  make  this  connection,  we  created  a  set 
of  intervening  variables  from  the  political  psychology  literature  that  help  express  the  rela¬ 
tionship  of  traits  to  actions.  The  following  six  intervening  variables  link  actions  to  personal¬ 
ity,  in  that  they  can  be  considered  to  be  both  action  characteristics  and  behavioral  proclivi¬ 
ties. 

•  Conflict  versus  cooperation  (regarding  opponents); 

•  Follow  through  required  versus  not  required; 

•  Consistent  with  position  versus  not  consistent; 

•  Unilateral  versus  collaborative  (regarding  colleagues); 

•  Substantive  versus  protocol;  and 

•  Challenges  constraints  versus  no  challenges. 

HumRRO,  along  with  Intelligence  Community  psychologists  estimated  the  quantita¬ 
tive  relationship  between  the  personality  traits  and  the  six  behavioral  proclivities;  these  cor- 
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relations  are  embedded  in  the  model.  When  we  model  a  particular  decision,  we  draw  depend¬ 
encies  between  hypothesis  (decision)  node  and  the  action  characteristics,  thus  specifying 
which  proclivities  are  relevant  to  the  decision.  The  process  recognizes  that  not  every  procliv¬ 
ity  relates  to  a  particular  set  of  hypothesized  actions. 

Model  Implementation  Process 

We  have  developed  models  in  two-day,  facilitated  meetings  attended  by  analysts, 
model  developers,  and  external  subject-matter  experts.  The  facilitator  guides  the  participants 
through  the  steps  in  the  development  process  (described  in  the  following  sections),  elicits 
estimates  of  model  parameters,  and  ensures  that  the  requirements  of  the  methodology  are 
met.  A  second  member  of  the  modeling  staff  implements  the  model  on  the  computer  and 
takes  notes.  The  model  is  projected  onto  a  screen  during  the  development  process  so  that  all 
participants  are  aware  of  the  variables  and  relationships  included  in  it.  Both  analysts  and  ex¬ 
ternal  experts  provide  the  information  and  assessments  that  are  incorporated  into  the  model. 
The  analysts  usually  provide  critical  information  about  the  questions  to  be  addressed  by  the 
model,  while  all  participants  provide  the  regional  and  Subject  knowledge  incorporated  into 
the  model. 

The  analysts’  intelligence  questions,  the  optional  outcomes  of  interest  -  what  will  X 
do  or  what  can  we  do  to  lead  X  to  do  Y  -  are  debated  at  considerable  length.  The  process  in 
2.3  below  usually  takes  four  hours  at  the  outset  of  a  two-day  session  and  sets  the  motif  -  and 
some  social  norms  -  for  the  rest  of  the  session.  The  mix  of  staff  and  outside  panelists  sug¬ 
gests  that  diversity  of  opinion  and  experience  is  desirable;  it  gives  the  staff  permission  and 
cover  for  bringing  new  or  divergent  views  to  the  table:  •  ' 

•  The  facilitator’s  behavior  is  critical  in  two  respects.  First,  the  technical  aspects  of  ap¬ 
plying  Bayesian  analysis  must  be  guided  by  an  expert  -  the  international  relations  and  politi¬ 
cal  science  educations  of  most  analysts  don’t  prepare  them  for  this  methodology.  The  chal¬ 
lenges  of  helping  the  group  to  frame  questions  properly  -  consistent  with  probability  the¬ 
ory  -  and  to  keep  their  engagement  fresh  while  estimating  large  conditional  probability  ta¬ 
bles  are  not  trivial  items.  In  addition,  the  facilitator  helps  to  keep  the  gate  open  to  contrary 
data  and  judgments  and  healthy  debate,  to  elicit  contributions  from  all  members,  to  challenge 
what  everyone  takes  for  granted,  and  to  curb  the  natural  tendencies  of  dominant  actors  to  hog 
the  stage  and  dictate  the  analysis  -  all  while  demonstrating  respect  for  each  contribution. 

As  the  session  proceeds  and  the  facilitator  leads  the  team  through  identifying  the  key 
determinative  predictors  and  indicators  of  the  situational  variables,  much  debate  about  key 
variables  ensues.  Projecting  the  model  on  a  screen  as  it  is  being  developed  provides  a  way  to 
focus  the  discussion  on  specific  issues,  data,  and  opinions,  while  avoiding  unproductive  ad 
hominem  debates.  In  addition,  it  may  provide  an  environment  that  can  encourage  greater  par¬ 
ticipation  from  reticent  analysts.  Consensus  may  not  be  feasible,  but  the  model  makes  it  pos¬ 
sible  to  locate  specific  areas  of  agreement  and  disagreement  and  to  determine  the  implica¬ 
tions  of  this  disagreement  on  the  outcome  of  the  model.  Where  there  is  disagreement  about 
critical  model  variables,  the  areas  of  disagreement  can  be  used  to  specify  the  requirement  for 
additional  intelligence  collection. 
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Throughout  the  session,  a  note  taker  records  choices,  issues  and  rationales  for  deci¬ 
sions  to  be  included  with  the  model  as  a  history,  which  can  help  future  users  understand  the 
logic  underlying  the  model.  At  the  end,  we  invite  the  group  to  review  the  model  after  a  day 
or  more  -  a  process  that  can  iron  out  wrinkles  and  spot  deficits.  Of  course,  many  other  social 
dynamics  are  managed  in  this  process,  but  these  are  the  highlights. 

We  now  illustrate  the  model  development  process  with  a  hypothetical  example.  Al¬ 
though  based  on  the  analysis  of  an  actual  event,  key  elements  and  assessments  of  the  model 
have  been  changed  so  that  it  may  be  presented  in  an  unclassified  format.  The  description 
covers  five  steps  in  the  model  development  process:  (a)  defining  the  question,  (b)  modeling 
the  situation,  (c)  adding  personality,  (d)  performing  “what-if’  analyses,  and  (e)  assessing 
sensitivity  of  variables. 

Defining  the  Question 

The  Subject’s  decision,  for  instance,  to  launch  an  attack  is  not  simply  yes-no,  but 
whether  to  make  a  contingent  attack,  one  involving  certain  levels  of  force,  on  certain  days, 
against  certain  targets,  or  seeking  certain  outcomes,  or  may  weigh  attacks  versus  warnings  or 
other  public  acts.  These  alternative  competing  decisions  represent  the  analyst’s  best  estimate 
of  the  choices  considered  by  the  Subject.  We  have  on  occasion  used  between  one  and  four 
variables,  each  having  two  to  six  states  to  define  the  possible  prediction  of  a  key  figure’s  de¬ 
cision.  These  states  (in  each  variable)  need  to  be  mutually  exclusive  and  collectively  exhaus¬ 
tive.  Clearly,  when  trying  to  predict  the  future  %y  using  a  discrete  number  (4  to  44)  of  states, 
we  must  interpret  mutually  exclusive  and  collectively  exhaustive  loosely.  For  the  sample 
problem  being  used  in  this  paper,  we  address  a  situation  in  which  a  leader  must  decide  what 
to  do  when  beset  by  a  national  strike  organized  by  his  opponents.  The  states  defined  by  the 
intelligence  analysts  were: 

•  Leave  the  country; 

•  Make  concessions  to  end  the  strike; 

•  Hold  a  voter  referendum  in  agreement  to  end  the  strike; 

•  Allow  a  regional  organization  to  arbitrate  the  strike; 

•  Wait  out  the  strikers;  or 

•  Repress  the  strikers  using  violence  when  necessary. 

We  also  define  the  leader’s  strategic  objectives  and  develop  a  probabilistic  relation¬ 
ship  between  the  objectives  and  the  hypothesized  actions  (see  Figure  2).  The  leader’s  objec¬ 
tives  typically  range  from  “staying  alive”  to  “becoming  an  international  power”.  These  states 
for  the  leader’s  objectives  are  clearly  not  mutually  exclusive  so  we  define  the  objectives 
variable  to  be  the  primary  objective  of  the  leader. 

The  two  boxes  in  Figure  2  represent  variables  (or  nodes)  that  may  take  one  of  several 
values  (or  states).  The  arc  from  Leader’s  Objective  to  Leader  Decn  in  Sec  Event  establishes 
that  there  is  probabilistic  dependence  between  these  two  variables. 
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Modeling  the  Situation 


When  a  model  is  being  developed,  significant  effort  is  made  to  identify  possible  situ¬ 
ational  variables  that  might  change  the  outcome  of  the  leader’s  decision.  Once  the  possible 
situational  events  have  been  discussed  and  prioritized,  key  events  are  picked  and  added  one 
at  a  time.  After  each  variable  is  added,  we  conduct  several  “what  if’  analyses  (changing  situ¬ 
ational  outcomes)  to  see  if  the  “model”  makes  sense  in  these  different  situations.  When  these 
analyses  reveal  errors  or  inconsistencies  in  the  predicted  probabilities,  appropriate  changes 
are  made  to  the  model.  Additional  variables  are  added  subject  to  the  time  constraints  for  the 
model  development  process.  Figure  3  shows  the  national  strike  model  with  the  added  situ¬ 
ational  variables  (one-and-a-half  days  of  the  two-day  session  are  typically  completed  by  this 
point.) 
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To  develop  the  Bayesian  network,  it  is  necessary  to  represent  the  dependencies  be¬ 
tween  variables  by  conditional  probabilities.  Table  2  shows  the  conditional  probability  table 
for  one  variable  -  Mgmt  of  Strike  Leaders.  There  are  similar  tables  for  every  node/variable  in 
the  model. 


Table  2.  Conditional  Probability  Table  - 
Probability  of  Mgt  of  Strike  Leaders  given  Leader  Decn 


f  Leader  Decn 

Me 

None 

t  of  Strike  Lead 
ConstJailFines 

ers 

UnconDetent 

LeaveCountry 

0.90 

0.01 

0.09 

Concessions 

0.60 

0.30 

0.10 

VoteRefern 

0.50 

0.30 

0.20 

UseRgnlOrg 

0.75 

0.20 

0.05 

WaitOut 

0.60 

0.30 

0.10 

ViolentRepress 

0.05 

0.20 

0.75 

Figure  4  shows  the  updated  probabilities  after  3  weeks  have  elapsed,  a  number  of  In¬ 
tel  reports  have  been  received,  and  the  values  of  some  of  the  situational  variables  are  known 
with  near  certainty,  as  shown  by  the  shaded  nodes  in  the  figure. 


Figure  4.  Updated  Model  after  3  Weeks 


Adding  Personality 


To  incorporate  the  Subject’s  personality  into  the  model,  the  analyst  must  assess  the 
Subject’s  position  on  personality  measures  and  relate  the  variables  representing  action  char- 
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acteristics  (or  behavioral  proclivities)  to  the  hypothesized  actions.  The  relationships  between 
measures,  personality  variables,  and  linking  variables  are  fixed,  based  on  previous  research, 
analyses  by  HumRRO  researchers,  and  estimates  by  Intelligence  Community  psychologists. 

Figure  5  shows  the  resulting  model  after  personality  was  incorporated.  The  situational 
model  (on  the  left)  is  connected  to  the  personality  model  (on  the  right),  using  four  of  the  six 
action  characteristics.  Each  link  between  the  hypothesis  node  and  one  of  the  action  character¬ 
istics  is  associated  with  a  conditional  probability  table  that  describes  how  the  hypothesized 
actions  differ  with  regard  to  that  specific  characteristic.  The  fact  that  two  characteristics  were 
not  linked  to  the  hypothesis  node  indicates  that  the  actions  did  not  differ  with  regard  to  those 
two  characteristics.  In  addition,  the  analysts  enter  as  much  personality  information  as  is 
available  on  the  Subject.  The  assessments  of  the  analysts  are  shown  by  the  probability  distri¬ 
butions  in  the  personality  nodes  on  the  right  side  of  Figure  5.  The  results  of  this  effort  mod¬ 
ify  the  action  probabilities  from  the  ones  that  were  based  on  situational  factors  only. 


Performing  “What  If’  Analyses 

“What  if’  analyses  are  defined  as  tests  of  a  model  made  by  setting  model  parameters 
to  see  (a)  how  the  changes  in  antecedent  variable  affects  the  outcome,  and  (b)  if  the  results  at 
that  setting  make  sense.  In  addition,  comparisons  are  made  across  multiple  “what  if’  tests  to 
see  how  changes  in  antecedents  affect  the  relative  results  of  the  model  from  different  settings 
and  how  they  may  make  relative  sense. 
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For  example,  we  would  move  the  probability  setting  on  one  of  the  situation  variables 
from  one  extreme  to  another  to  observe  the  impact  on  the  probabilities  of  the  hypothesis.  We 
would  also  compare  the  results  of  this  analysis  to  those  for  other  variables,  first  singly  to  ob¬ 
serve  the  relative  magnitudes  of  effect  one  variable  makes  with  respect  to  the  others.  Then 
we  compare  the  effects  introduced  by  interactions  among  the  situation  variables.  When  we 
find  results  that  do  not  make  sense,  we  check  for  changes  in  the  probability  tables  that  would 
produce  the  results  the  analysts  feel  make  sense.  Sometimes,  the  analysts  desire  to  make  the 
changes  to  the  probability  tables;  other  times,  they  prefer  to  leave  the  probability  tables  as 
they  were  since  they  make  more  sense  than  the  desired  results.  The  software  is  very  flexible 
in  handling  these  calibrations  and  instantaneous  in  revealing  results  of  multiple  “what  if’ 
propositions. 

Assessing  the  Sensitivity  of  Variables 

In  most  Bayesian  network  software  implementations,  the  user  can  designate  a  node 
and  calculate  the  mutual  information  between  the  selected  node  and  other  nodes,  one  at  a 
time.  This  calculation  identifies  (based  on  the  mutual  information  metric)  the  relative  impact 
that  changes  in  the  probabilities  of  other  notes  will  have  on  the  probabilities  of  the  desig¬ 
nated  node. 


Summary  and  Conclusions 

In  the  last  two  years  we  have  built  the  following  models  while  working  with  teams  of 
intelligence  analysts  and  expert  consultants: 

•  Invasion 

•  National  strike 

•  Domestic  threat  * 

•  Missile  testing 

•  Support  for  the  Global  War  on  Terrorism 

•  Dispute  over  contested  territory 

•  Peace/cease-fire  negotiation  * 

•  Use  of  WMD  * 

•  Monetary  devaluation  * 

•  Establishment  of  a  new  caliphate* 

•  Operational  planning  in  a  terror  cell* 

Those  models  marked  with  an  asterisk  (*)  are  forward-looking  models  for  which  the 
answer  was  not  known  when  the  model  was  built.  The  models  vary  in  many  ways: 

1.  The  number  of  hypothesis  nodes  (discussed  earlier), 

2.  The  number  and  complexity  of  causal  relations  from  nodes  addressing  US  and  other 
major  country  actions  to  the  hypothesis  nodes, 

3.  The  number  of  relations  between  perceived  reactions  of  the  US  and  other  major  pow¬ 
ers  and  the  hypothesis  nodes,  and 
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4.  The  number  and  complexity  of  indicator  variables,  that  have  arcs  entering  them  from 

the  hypothesis  nodes 

We  plan  to  compare  the  predictions  of  these  models  to  actual  events  to  estimate  the 
validity  of  the  models.  The  models  without  asterisks  were  post-dictions  of  historical  events. 
Although  these  models  cannot  be  validated  in  the  same  sense  as  the  forward-looking  models, 
we  will  investigate  the  extent  to  which  a  model  of  a  historical  event  can  be  applied  to  a  simi¬ 
lar  situation  with  a  different  Subject. 

Some  of  these  models  have  proven  very  useful  and  resulted  in  published  papers  for 
consumers  within  the  intelligence  community.  Other  models  have  less  useful.  No  group  has 
said  the  model  building  process  was  a  waste  of  time. 

The  APOLLO  program,  underway  for  three  years  now,  is  currently  delivering  a  soft¬ 
ware-based  tool  to  intelligence  analysts  that  supports  the  development  of  Bayesian  network 
models  to  address  a  wide  range  of  situations  in  which  a  leader  is  making  a  decision,  the  ef¬ 
fects  of  which  will  evolve  over  several  weeks/months.  A  library  of  models  has  been  under 
development  during  this  time  period  as  a  proof  of  concept  and  as  a  resource  for  analysts  to 
use  as  part  of  bootstrapping  their  efforts.  The  models  span  many  different  topic  areas  (inva¬ 
sions,  national  strikes,  missile  testing,  weapons  of  mass  destruction,  and  economics). 

Automating  the  Integration  of  Reports 

In  future  research,  the  Bayesian  network  will  be  tied  into  AIPSA  (Automated  Intel 
Processing  for  Situational  Awareness)  being  developed  by  the  Pacific  Northwest  National 
Laboratory  (PNNL),  which  is  based  on  the  INSPIRE  engine.  AIPSA  will  sort  through  incom¬ 
ing  all-source  intelligence  reports  and  Web  documents  using  the  analysts’  favorite  search  en¬ 
gine,  rank  their  relative  salience  to  each  situational  variable,  and  captures  the  source  infor¬ 
mation  for  later  user  and  management  review.  The  analyst  then  updates  -  manually  or  with 
partial  automation  (available  in  2006)  -  the  predictive  model.  As  a  result  of  the  evidence 
changing  the  model,  the  APOLLO  software  will  recompute  the  probabilities  and  alert  the 
analyst  or  others  when  the  outcome  probability  crosses  a  user-selected  threshold  based  on 
new  evidence. 

To  support  the  production  process  and  to  add  efficiency,  the  evidence  (source  docu¬ 
ments)  supporting  each  version  of  the  Bayesian  network,  the  entire  model,  the  probabilities 
associated  with  the  variables,  and  any  annotations  the  analyst  makes  are  stamped  with  date 
and  user  and  stored  for  future  editing  or  reference. 

Updating  the  Personality  Model  and  Adding  Culture 

Although  the  personality  model  provides  a  very  general  framework  for  representing 
relationships  between  leader  personality  and  behavioral  proclivities,  the  relationships  repre¬ 
sented  within  this  framework  were  based  on  the  judgments  of  a  relatively  small  number  of 
experts.  We  are  currently  completing  an  expert  judgment  study  that  provides  a  more  compre¬ 
hensive  view  of  the  relationship  between  personality  and  behavioral  proclivities.  The 
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strength  of  these  relationships  was  assessed  by  more  than  40  experts  on  leadership  in  politi¬ 
cal,  military,  and  business  settings.  Based  on  the  results  of  the  study,  we  will  revise  both  the 
content  and  the  links  included  in  the  personality  model. 

In  addition,  we  are  conducting  research  to  represent  cultural  factors  in  the  Bayesian 
networks.  Our  initial  work  represents  differences  in  leader  personality  distributions  as  a 
function  of  national  culture.  Future  research  will  investigate  other  links  between  culture  and 
leader  decisions,  such  as  direct  links  between  cultural  variables  and  behavioral  proclivities. 

Improving  the  Model  Building  Process 

There  are  tentative  plans  to  develop  to  add  to  the  current  model  building  process, 
which  consists  of  reviewing  past  models  and  template  models  of  situations  addressed  in  the 
past.  These  tentative  plans  include  developing  an  “expert  model  critiquer”  that  could  be  en¬ 
gaged  by  analysts  to  review  a  draft  model  and  suggest  areas  where  improvement  might  be 
made  or  more  work  might  prove  fruitful. 

Providing  an  Explanatory  Capability  for  Specific  Results  of  the  Model 

There  are  tentative  plans  to  build  upon  limited  research  in  the  Bayesian  network 
community  and  develop  the  capability  to  provide  verbal  explanations  for  the  analyst  to  use  in 
reporting  as  a  means  of  “explaining”  why  specific  model  results  are  what  they  are. 
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